Papers with GRPO training
Predicate-Guided Generation for Mathematical Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that Prolog-MATH generates 81.3% solution coverage on Deepseek-V3 . |
| Approach: | They propose a curated corpus to support mathematical reasoning in large language models . they propose supervised fine-tuning followed by GRPO training to address problems that Deepseek-V3 fails to solve. |
| Outcome: | The proposed pipeline achieves 81.3% solution coverage on the Deepseek-V3 training set. |
MMR-GRPO: Accelerating GRPO-Style Training through Diversity-Aware Reward Reweighting (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have demonstrated remarkable capabilities in mathematical reasoning tasks. |
| Approach: | They propose to use Maximal Marginal Relevance to reweigh rewards of multiple rollouts by balancing rollout quality with diversity to reduce rollout redundancy. |
| Outcome: | The proposed approach reduces training time and costs by 47.9% . evaluations across three model sizes, three GRPO variants, and five mathematical reasoning benchmarks show that it achieves comparable peak performance while requiring on average 70.2% less wall-clock time. |
QA‐LIGN: Aligning LLMs through Constitutionally Decomposed QA (2025.findings-emnlp)
Copied to clipboard
Jacob Dineen, Aswin Rrv, Qin Liu, Zhikun Xu, Xiao Ye, Ming Shen, Zhaonan Li, Shijie Lu, Chitta Baral, Muhao Chen, Ben Zhou
| Challenge: | QA-LIGN decomposes monolithic rewards into interpretable principle-specific evaluations . scalar rewards obscure which objectives drive the training signal . |
| Approach: | a new method decomposes monolithic rewards into interpretable principle-specific evaluations . QA-LIGN reduces attack success rates by up to 68.7% while maintaining a 0.67% false refusal rate . |
| Outcome: | QA-LIGN reduces attack success rates by up to 68.7% while maintaining a 0.67% false refusal rate . the results outperform DPO and GRPO with state-of-the-art reward models given equivalent training . |